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ABSTBACT 

The results of the classroom observations of students 
during compeiisatory reading instruction are described • Sixty-three 
second-, fourth-, and sixth-grade classrooms across the country were 
observed usiag the Student Observation Scale. The scale is 
two-dimensional enabling an observer to code simultaneously two 
.different aspects of an observed activity. One dimension. Group, 
contained four categories while the other dimension. Content, 
contained 12 categories. Therefore, the scale spanned 48 
two-dimensional categories. Besults are reported in terms of percent 
of time students spent in each of the categories of the Student 
Observation Scale. (Author) 
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The Classroom Behavior of Students 
During Compensatory Reading Instruction* 

. ^ , Susan F. Weinberg 

Katherine B. Nalin 
Thomas J. Quirk 

This report describes the results of the classroom observations of students 
during compensatory reading instruction.** The Student Observation Scale — 
Reading was designed to describe the major types of interaction during reading 
instruction so that a systematic study of students' activities during compensa- 
tory reading instruction could be undertaken. 

From the student's point of view, what is important to his development in 

i' 

reading ability is not only the activities which tlie teacher is performing, but 
also those activities which become his personal experiences; in short, those 
stimuli 1:o which the student is attending in the classroom become the focus of 
his particular learning experiences. For example > if the teacher is presenting 
an explanation to the class dealing with the comprehension of a paragraph, 
while several of the students are working math problems at their desks, it is 
the mathematics problems that become the focus of the learning behavior of 
these students, and not the teacher's comprehension activities. 

*Paper presented at the American Educational Research Association Convention, 
Chicago, 111., April 1974. 

**This research was conducted as part of an ETS contract with the U.Sr Office 
of Education, OEO-0-71-3715. 
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SELECTION OF THE PROGRAMS TO BE OBSERVED 

In July 1971, Educational Testing Service was requested by the U.S, Office 
of Education to develop design and analysis plans for a study of compensatory 
reading programs in U.S. public schools. A three phase study was conceptualized 
beginning with a Spring 1972 questionnaire survey of compensatory reading 
programs offered in grades two, four, and six of the U.S. public schools. In 
Phase II of the project, data descriptive of compensatory and non-compensatory 
reading programs were again obtained via questioi.aaires similar to those ad- 
ministered In the Spring of 1972. In addition. Fall 1972 and Spring 1973 student 
scores on measures of reading achievement and attitudes were obtained. The study 
of the classroom behavior of students, described in this report, was embedded 
in this second phase of the Compensatory Reading Project. As part of the Phase 
II study a small group of compensatory reading programs were selected which were 
considered innovative by persons knowledgeable in the field of reading education • 
Reading consultants and members of the Advisory Board of the Compensatory 
Reading Project were requested to identify specific compensatory reading 
programs they considered innovative, and to submit names of other reading 
experts who were qualified to do the same. In addition, testing directors of 
several large city school systems were asked to nominate innovative programs. 
Finally, the U.S. Office of Education selected a small number of states whose 
Directors of Title! Programs were requested to submit nominations. It was 
decided that programs would be selected in order to obtain a wide variety of 
instructional characteristics. Program effectiveness was not a criterion of 
selection, since in most cases no adequate data by which to judge effectiveness 
were available. Phone interviews and site visits were conducted and the end 
result of this sequence of nomination, phone interview, and site visit was the 
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selection of 34 schools in 21 cities, in which a total of 156 classes were 
observed. Sixty-seven second, fourth, and sixth grade classrooms across the 
country from 13 schools in 10 cities were selected to be observed using the 
Student Observation Scale — Reading described in this report. 

Table 1 presents the reading programs found in each school and the number 
of classrooms observed in each school. 

DEVELOPMENT OF THE OBSERVATION SCALE 

The categories of the Student Observation Scale — Reading were designed to 
apply to student reading activities whether or not these activities took place 
in compensatory reading classes. The members of the research team visited a 
number of second, fourth, and sixth grade reading <?lasses and kept a log of the 
verbal and non-verbal activities which took place during these visits. Each 
activity 'was then written on a separate index card so that these cards could be 
used later to aid in classifying the activities into categories of similar 
behaviorp. The resulting categories were pilot tested and revised until the 
Student Observation Scale — Reading finally included two dimensions: (1) the 
Group of Instruction, and (2) the Content of Instruction. 

The Group dimension described the type of group the student was paying 
attention to. The Content dimension described the kind or type of activity 
being performed by the student. The main advantage of using this two- 
dimensional coding scale was that an observer could code simultaneously both 
the Group and the Content of an activity being observed in the classroom. 
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This observation Instrument was designed to be used with a stopwatch so 
that student activities in the classroom could be coded in fixed-time intervals. 
The observation procedure required the observer to watch a different student 
during each 15-second interval and to decide what activity each student was 
engaged in as the sweep-hand of the stopwatch completed each 15-second interval* 
A 15-second interval was selected after pilot tests indicated that the observers 
required this much time in order to be able to code reliably a different student 
during each time interval. During each visit in every classroom the observer 
coded the behavior of a different student every 15 seconds until the whole 
class had been observed at which time the procedure was repeated until a total 
of 15 minutes of coding by the observer had taken place. 

All of the student observers used a coding foj;m which included the Group 
categories denoted by four columns (Teacher, Other Adult, Peer, Alone) and the 
twelve. Content categories denoted by the numbers 1-12. The observed activity 
was represented by placing the number of the Content category which described 
the observed acti.vity into the appropriate Group column on the coding sheet. 
A sample coding sheet is presented in Table 2. 

DEFINITIONS OF THE CATEGORIES USED TO CODE STUDENT BEHAVIOR 

Brief definitions and examples of each of the coding categories are presented 
in the following section of this report, and more complete descriptions are 
available elsewhere (Quirk, Weinberg, & Nalin, 1973). 

The Group of Instruction 

The Group of Instruction refers to the type of classroom setting or group 
in which the activities of the student being observed take place. The four 
Groups of Instruction are: Teacher, Other Adult, Peer, and Alone. 

ERLC 
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Table 2 



Coding Sheet for Coding Group - Content Combination 

Name of Observer 

Date 



Teacher 



Time Start 



Finish 



Third of Class 1 2 3 SP 

School " 

Grade • 

City 



Reading 
1 • Comp • 

2. Pron. & Wd. Rec, 

3. Lang* Struc. 

4. Rdg. Silently 



Content 
Other Instructional 

5. Sp. 

6. Writing 

7. List. Inst. 

8. Non-Rdg. Inst. 

9. Man. Inst. 



Other 

10. Pos. Fdbk. 

11. Neg. Fdbk. 

12. Ex. 



Group 



Teacher 



Other Adult 



Peer 



Alone 
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The "Teacher" Group of Instruction refers to those instances in which the 
student being observed is paying attention to the teacher or is paying attention 
to a group that includes the teacher. (Example: The student being observed is 
watching the teacher who is reading a story aloud to the class.) 

The "other Adult" Group of Instruction refers to those instances in which 
the student being observed is paying attention either to an adult other than 
the teacher or to a group that contains an adult other than the teacher. 
(Example: An aide is telling the student being observed how to spell the word 
trough .) 

The "Peer" Group of Instruction refers to those instances in which the 
student being observed is paying attention to a group which contains his peer 
or peers, but does not contain either the teacher or an other adult. (Example: 
The student being observed is reading a story aloud to another student.) 

The "Alone" Group of Instruction refers to those instances in which the 
student being observed is not paying attention to any group, teacher, other 
adult, or peer in the classroom, but is paying attention to his own thoughts, 
to his own set of materials, or to a machine when he Is by himself. (Example: 
The student being observed is doing spelling exercises alone at his desk*) 

Whenever the student being observed was paying attention to a group that 
contained both the teacher and an other adult, the observers were trained to 
code this situation as the Teacher Group even if the other adult was speaking. 
When the student being observed was paying attention to the observer the 
observers were instructed to code these Instances as Alone. 



The Content of Instruction 

The Content of Instruction refers to the type of instructional or non- 
instructional activity in which the student being observed is engaged. There 
were twelve categories used to describe the Content of Instruction • Four of 
these categories are related specifically to reading activities: Comprehension, 
Pronunciation and Word Recognition, Language Structure, and Reading Silently, 
For the purposes of this observation instrument, reading activities referred 
only to those activities in the classroom which included a printed stimulus 
in the form of letters of the alphabet or combinations of letters, words, 
phrases, sentences, and paragraphs. 

Eight other categories were used to describe other instructional activities 
Spelling, Writing, Non-Reading Instruction, Listening Instruction, Management 
Instruction, Positive Feedback, Negative Feedback, and Extraneous. Brief 
definitions and one example of each of the Content categories will now be 
presented. 

Category 1; Comprehension . This category refers to those instances in 
which the teacher, students, machines, or other persons ir the classroom 
demonstrate understanding of what the students have read. It includes questions 
statements, or actions such as defining a word, giving the meaning of a sentence 
or interpreting a story. (Example: The student being observed says, "In that 
story. Bob liked the lion.") 

Category 2: Pronunciation and Word Recognition . This category refers to 
those instances when some person or a machine is pronouncing aloud letter 
combinations, words, phrases, sentences, or stories which the student being 
observed can see or is reading. This category also includes phonic rules which 
deal with pronunciation symbols and rules for vowel, consonant, and combination 
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sounds. In addition, this category includes non-verbal actions such as 
pointing, writing, coloring, and the like that persons or machines in the 
classroom use to indicate questions or answers to Pronunciation and Word 
Recognition problems. (Example: The student being observed asks, "How do ' 
you pronounce this word?") 

Category 3: Language Structure . This category refers to the structure 
of a word, phrase, sentence, or paragraph where the student being observed has 
read or seen the printed material being discussed. It involves punctuation, 
grammatical construction, and syllabification when it is done for the purpose 
of hyphenating a word. (Example: The teacher asks the student who is being 
observed, "What kind of punctuation follows a question?") 

Category 4: Reading Silently . This category refers to all instances in 
which the student being observed is reading silently to himself or is silently 
looking at printed material. (Example: The student being observed is silently 
reading a story at his desk.) 

Category 5: Spelling . This category refers to those instances in which 
words or parts of words are formed one letter at a time, aloud, to oneself, on 
the board, or on a sheet of paper. It also includes activities which have to 
do with individual letters of the alphabet and alphabetizing activities. The 
dictation of words or sentences by the teacher or students so that the students 
can write out these words are also included as spelling activities. (Example: 
The student being observed asks, "How do you spell the word pluck? ") 

Category 6: Writing . This category refers to those specific activities 
in which the student being observed is creating or composing his own original 
work in terms of words, phrases, or sentences. This category does not include 
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copying words > phrases, or sentences from the board or from a book. This 
category also refers to writing activities such as the teacher writing a story 
which the student being observed Is making up and dictating to her. (Example; 
The student being observed is writing a story about what he did last summer.) 

Category 7; Listening Instruction . This category includes questions,' 
statements, and actions which refer to reading material that has been or will 
be read to the student being observed, but which the student himself has not 
seen or read. Also included in this category are instances in which some person 
is reading aloud to the student being observed when the material that is being 
read cannot be seen by the student* (Example: The student being observed 

listens as the teacher reads him a story. The student does not have any books 

> 

open in front of him.) 

Category 8; Non-Reading Instruction . This category includes those 
activities which are instructional in content but which are not specific reading 
activities or activities which refer to material that has been read to the 
students. (Example: The teacher says to the student being observed, "See the 
new growth on this piece of moss that I brought in to show you.") 

Category 9: Management Instruction . This category includes those instances 
in which the student being observed is paying attention to transitional activities, 
managing activities, or directing activities performed by persons or machines 
in the classroom that facilitate the instruction taking place during reading or 
other instructional activities. (Example: The teacher says to the student 
being observed, "Debby, your group is ready to come over here, so come on over.") 

Category 10: Positive Feedback * This category includes those instances 
in which the student being observed is paying attention to actions or words by 
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the teacher, other adult, or machine that praise or encourage the activities of 
himself or others in the classroom. (Example: The teacher says to the student 
being observed, "Very good, John.") 

Category lit Negative Feedback . This category includes those instances 
in which the student being observed is paying attention to actions by the teacher, 
other adult, or a machine that indicate to a person in the classroom that his 
answer is wrong or that discipline persons in the classroom for their actions. 
(Example: The teacher says, "Class, be quiet.") 

Category 12: Extraneous . This category includes those instances in which 
the student being (observed is paying attention to irrelevant or incidental 
comments or actions which are not codeable into any .'other category » (Example: 
The student being observed asks the teacher to sign his late slip.) 

These Content categories required several special coding rules that were 
necessary to permit the observers to recognize the categories as discrete. 
These coding rules are described in detail elsewhere (Quirk, Weinberg, & Nalin, 
1973), but one of them is worth noting. This rule dealt with a "carry-over 
effect" in which pauses in instruction (e.g., whenever the teacher was waiting 
for a student to answer her question) were coded as part of the Content category 
that was the focus of the instruction. For example, if the teacher asked the 
students how to pronounce a word written on the board, and was waiting for a 
response from the class when the stopwatch indicated that this event was to be 
coded, thiL pause would be coded as Content category 2 (Pronunciation and Word 
Recognition) . 
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TRAINING OF OBSERVERS 

A. total of 27 observers (6 men and 21 women) were trained to use the 
Student Observation Scale — Reading. They ranged in age from 22 to 71 years 
with a median age of 36 years. 

A trainer and an assistant trainer conducted three separate one-week 
training programs for the Student Observation Scale — Reading. This three-week 
period lasted from October 15 through November 3, 1972. The training programs 
were located in Princeton, New Jersey; Evanston, Illinois; and Berkeley, 
California. Each training session lasted for five days. The total time spent 
in training the observers during each training session was approximately 35 
hours. 

A reliability study was conducted on the last day of each training session. 
The procedure was to have the observers observe a reading class in pairs and 
to use a single stopwatch to synchronize their codings. These pairs of observers 
coded for a 20-minute observation period. Each observer was then paired with a 
different observer for a second observation period and still another for a 
third observation period. Twenty-one of the observers were paired in this manner 
while six of the observers completed only two observation periods of 20 minutes 
each. 

One of the observers was not permitted to collect any classroom observation 
data during the study because her reliability coefficients were judged to be 
too low. For the remaining observers, their reliabilities, when computed by 
using the total codings for all of the observers with whom they were paired, 
were as follows: the Group of Instruction ranged from .81 to 1.0 with a median 
coefficient of .96; for the Content of Instruction, the reliabilities ranged 
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from .67 to .99 with a median coefficient of .90; for the Groups-Content dimensions 
combined into a two-dimensional set of categories (4 Group categories x 12 
Content categories), the reliabilities ranged from .62 to .99 with a median 
coefficient of .86. All reliabilities reported are Scott's ir coefficients 
(Scott, 1955). 

DATA COLLECTION 

The classroom observation data for the Student Observation Scale — Reading 
were collected from October 26, 1972 until February 8, 1973. A total of 67 
classes in 13 schools in 10 cities were selected to be observed. There were 41 
second grade classes, 18 fourth grade classes, and eight sixth grade classes 
selected to be observed^^ Each class was supposed to be observed on nine separate 
days for 15 minutes of coding each day for a total of 2 1/4 hours of observation 
of each class. The observers were asked to distribute these nine observations 
so that three of them took place during the first third of the reading period, 
three of them took place during the middle third of the reading period, and three 
of them took place during the last third of the reading period. 

There were nine observations completed on 63 of these 67 classes (94 percent 
completion rate). Four classes were not included for a variety of reasons. In 
one city, four classes which were observed nine times each were combined into 
one class since the teacher met with the same students for approximately two and 
one-half hours every day. One other teacher's class was not included because 
this teacher taught other teachers' classes as well as her own class and there 
were not enough observations of this teacher instructing her designated 
students. 
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DISCUSSION OF RESULTS 

The results of the classroom observations of the student behavior for the 
63 compensatory reading classes are summarized in Table 3. In Table 3, the 
percent of time spent in each category appears in the first row of each cell; 
this percentage was computed by dividing the total frequency of occurrence of 
each category across the classes by the total number of codings across all of 
the classes (35,292 codings). The resulting number represents the average 
percent of time spent in each category across all of the visits to these compensatory 
reading classes. In Table 3, the standard deviation across classes of the percent 
of time spent in each category by each class appears in parentheses in each cell. 
The rank order of the percent of time for each subcategory within its corresponding 
major category appears in the box in the lower rigKc corner of each cell. 

A total of 540 events were supposed to be coded for each class (9 visits 
per class x 15 minutes of coding per visit x 4 codings per minute). In fact, 
the number of events coded per teacher's class ranged from 504 to 720 events 
with a mean of 560.19 events and a standard deviation of 59.84. 

In one city the teachers met privately with small groups of students in a 
conference room. In this city, student behavior was coded for a total of 20 
minutes for each of nine visits instead of the usual 15 minutes of coding and 
in these eight classes the observers coded an average of 715 events per class 
instead of the usual 540 events. 

Student Behavior During Reading Instruction 

On the average, the students spent 48% of their time attending to the 
teacher or a group containing the teachev, 31% of their time alone, 12% of their 
time attending to some adult other than the teacher, and 9% of their time 
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attending to other students. Thus, the students were paying attention either 
to the teacher or to some other adult for 60% of their time. The fact that the 
students were not paying attention to any adult or any other student for almost 
one-third of their time is notable in that it indicated that the students 
exercised considerable control over their own thoughts and activities for a 
significant amount of the time. 

In terms of the Content of Instruction, the students spent the largest 
amount of their time in Pronunciation and Word Recognition activities (20%). 
The students spent 14% of their time in Extraneous activities, 14% of their time 
in Spelling activities, 13% of their nime attending to Management Instruction 
activities, and 13% of their time Reading Silently. The students spent 9% of 
their time in instructional activities which were not related to reading and 7% 
of their time in Comprehension activities. The students spent 4% or less of 
their time in the remaining Content categories. 

The students spent 61% of their time in activities which can be considered 
directly related to instruction in reading: Comprehension, Pronunciation and 
Word Recognition, Language Structure, Reading Silently, Spelling, Writing, and 
Listening Instruction. 

The students evenly distributed their time in attending to Positive Feedback 
(1.7%) and Negative Feedback (1.3%). 

\flien the Group-Content activities of the students were combined into two- 
dimensional categories, the students spent at least 3% of their time in each of 
the 11 categories summarized in Table 4. These 11 categories represented 71% 
of the students' time. With respect to Group of Instruction for these 11 
categories, the students distributed these activities by attending to the teacher 
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• Table A 

GROUP-COOTENT CATEGORIES OF STUDENT BEHAVIOR 
THAT OCCURREd AT LEAST 3% OF THE TIME 





Rank Order 


Category 


Percent of Time 



1 


Teacher: Pronunciation and Word Recognition 


13.41 


2 


Teacher; Management Instruction 


8.35 


3 


Alone: Reading Silently 


7.99 


4 


Alone: Extraneous 


7.95 


5 


Teacher: Spelling 


6.25 


6 


Teacher: Comprehension 


5.26 


7 


Alone: Spelling 


5.22 


8 


Peer: Extraneous 


5.16 


9 


Teacher: Reading Silently 


4.32 


10 


Teacher: Non-Reading Instruction 


3.60 


11 


J.- 

Teacher: Listening Instruction 


3.10 



Total 70.61 





Sumjoiary of Group Dimension 
for These lEleven Categories 


Summary of 
for These 


Content Dimension 
Eleven Categories 



Teacher 


44.29% 


Pronunciation & Word Recognition 


13.41% 


Alone 


21.16% 


Comprehension 


5.26% 


Peer 


5.16% 


Reading Silently 


12.31% 






Spelling 


11.47% 






Listening Instruction 


3.10% 






Non-Reading Instruction 


3.60% 






Management Instruction 


8.35% 






Extraneous 


13.11% 



o 
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for 44% of the time, by being alone for 21% of the time, and by attending to 
other students for 5% of the time. In terms of the Content of instruction for 
these 11 categories, the students spent 13% of the time in Pronunciation and 
Word Recognition activities, 13% of the time in Extraneous activities, 12% of 
the time Reading Silently, and 11% of the time in Spelling activities. 
Furthermore, the students spent 8% of the time in Management Instruction 
activities, 5% of the time in Comprehension activities, 4% of the time in Non- 
Reading Instruction activities, and 3% of the time in Listening Instruction 
activities, 

SUMMARY AND CONCLUSIONS 

The description of student behavior during cqppensatory reading instruction 
is a necessary first step in the systematic study of the relationship between 
what the students do during reading instruction and the subsequent development 
of the students in reading ability. How much time should teachers and students 
spend in activities involving Pronunciation and Word Recognition? Do those 
classes in which students spend more time in Comprehension activities improve 
more in reading ability? What is the relationship between the amount of time 
spent in Spelling activities and the students' improvement in reading scores? 
Questions such as these can only be answered by relating student behavior during 
reading instruction to student residual gain scores. This line of research 
may prove fertile in generating hypotheses about the relationship between 
student behavior and the students' subsequent development in reading ability. 
This research is currently being pursued at Educational Testing Service. The 
results are not yet available. 
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